Progressive and degenerative journals: on the growth and appraisal of knowledge in scholarly publishing.

Despite continued attention, finding adequate criteria for distinguishing "good" from "bad" scholarly journals remains an elusive goal. In this essay, I propose a solution informed by the work of Imre Lakatos and his methodology of scientific research programmes (MSRP). I begin by reviewing several notable attempts at appraising journal quality - focusing primarily on the impact factor and development of journal blacklists and whitelists. In doing so, I note their limitations and link their overarching goals to those found within the philosophy of science. I argue that Lakatos's MSRP and specifically his classifications of "progressive" and "degenerative" research programmes can be analogized and repurposed for the evaluation of scholarly journals. I argue that this alternative framework resolves some of the limitations discussed above and offers a more considered evaluation of journal quality - one that helps account for the historical evolution of journal-level publication practices and attendant contributions to the growth (or stunting) of scholarly knowledge. By doing so, the seeming problem of journal demarcation is diminished. In the process I utilize two novel tools (the mistake index and scite index) to further illustrate and operationalize aspects of the MSRP.

Causal and Associational Language in Observational Health Research: A Systematic Evaluation.

We estimated the degree to which language used in the high-profile medical/public health/epidemiology literature implied causality using language linking exposures to outcomes and action recommendations; examined disconnects between language and recommendations; identified the most common linking phrases; and estimated how strongly linking phrases imply causality. We searched for and screened 1,170 articles from 18 high-profile journals (65 per journal) published from 2010-2019. Based on written framing and systematic guidance, 3 reviewers rated the degree of causality implied in abstracts and full text for exposure/outcome linking language and action recommendations. Reviewers rated the causal implication of exposure/outcome linking language as none (no causal implication) in 13.8%, weak in 34.2%, moderate in 33.2%, and strong in 18.7% of abstracts. The implied causality of action recommendations was higher than the implied causality of linking sentences for 44.5% or commensurate for 40.3% of articles. The most common linking word in abstracts was "associate" (45.7%). Reviewers' ratings of linking word roots were highly heterogeneous; over half of reviewers rated "association" as having at least some causal implication. This research undercuts the assumption that avoiding "causal" words leads to clarity of interpretation in medical research.

A credit-like rating system to determine the legitimacy of scientific journals and publishers.

The predatory nature of a journal is in constant debate because it depends on multiple factors, which keep evolving. The classification of a journal as being predatory, or not, is no longer exclusively associated with its open access status, by inclusion or exclusion on perceived reputable academic indexes and/or on whitelists or blacklists. Inclusion in the latter may itself be determined by a host of criteria, may be riddled with type I errors (e.g., erroneous inclusion of a truly predatory journal in a whitelist) and/or type II errors (e.g., erroneous exclusion of a truly valid scholarly journal in a whitelist). While extreme cases of predatory publishing behavior may be clear cut, with true predatory journals displaying ample predatory properties, journals in non-binary grey zones of predatory criteria are difficult to classify. They may have some legitimate properties, but also some illegitimate ones. In such cases, it might be too extreme to refer to such entities as "predatory". Simply referring to them as "potentially predatory" or "borderline predatory" also does little justice to discern a predatory entity from an unscholarly, low-quality, unprofessional, or exploitative one. Faced with the limitations caused by this gradient of predatory dimensionality, this paper introduces a novel credit-like rating system, based in part on well-known financial credit ratings companies used to assess investment risk and creditworthiness, to assess journal or publisher quality. Cognizant of the weaknesses and criticisms of these rating systems, we suggest their use as a new way to view the scholarly nature of a journal or publisher. When used as a tool to supplement, replace, or reinforce current sets of criteria used for whitelists and blacklists, this system may provide a fresh perspective to gain a better understanding of predatory publishing behavior. Our tool does not propose to offer a definitive solution to this problem.